{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Synthethic data generation\n",
    "\n",
    "You can use LLMs to generate training data for fine-tuning LLMs. As you migh expect, the generated data is not as good as the real data, but it can be useful to bootstrap your data for fine-tuning LLMs. In this tutorial, we provide easy and simple examples to generate synthetic data using LLMs, but given the architecture of `distilabel` it is easy to scale this to way more complex pipelines and larger workloads."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "# install from develop because of small bug EvolQuality distilabel<1.2\n",
    "%pip install \"git+https://github.com/argilla-io/distilabel.git@develop#egg=distilabel[openai]\" -qq "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load dataset\n",
    "\n",
    "We will use the dataset from our [Data is Better Together](https://github.com/huggingface/data-is-better-together). Data is Better Together is a collaboration between 🤗 Hugging Face, 🏓 Argilla, and the Open-Source ML community. We aim to empower the open-source community to build impactful datasets collectively.\n",
    "\n",
    "This prompt ranking dataset was created by applying human evaluation in prompt, where roughly 400 people annotated human and synthehtic prompt to asses their quality on a scale from one to 5. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['prompt', 'quality', 'metadata', 'avg_rating', 'num_responses', 'agreement_ratio', 'raw_responses', 'kind', 'cluster_description', 'topic'],\n",
       "        num_rows: 10331\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset = load_dataset(\"DIBT/10k_prompts_ranked\")\n",
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'prompt': 'Provide step-by-step instructions on how to make a safe and effective homemade all-purpose cleaner from common household ingredients. The guide should include measurements, tips for storing the cleaner, and additional variations or scents that can be added. Additionally, the guide should be written in clear and concise language, with helpful visuals or photographs to aid in the process.',\n",
       " 'quality': [{'user_id': 'd23b12c2-b601-490e-b5b3-2040eb393a00',\n",
       "   'value': '4',\n",
       "   'status': 'submitted'},\n",
       "  {'user_id': 'e2bdd868-f28e-46fc-9254-a6ec1e291889',\n",
       "   'value': '4',\n",
       "   'status': 'submitted'}],\n",
       " 'metadata': '{\"source\": \"ultrachat\", \"kind\": \"synthetic\", \"evolved_from\": null}',\n",
       " 'avg_rating': 5.0,\n",
       " 'num_responses': 2,\n",
       " 'agreement_ratio': 1.0,\n",
       " 'raw_responses': [5, 5],\n",
       " 'kind': 'synthetic',\n",
       " 'cluster_description': 'Sustainable Packaging & Skin Care Products',\n",
       " 'topic': 'Environmental Issues'}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load LLMs\n",
    "\n",
    "We will now load a LLM integration within distilabel. For ease, we will use the `OpenAILLM`. Practically, there are two things to consider.\n",
    "\n",
    "- You can customize use any propietary or open-source LLM you want to use for vendor lock-in and licensing reasons.\n",
    "- Each LLM integration has its own arguments which are inherited from the original LLM provider.\n",
    "- You might want to use different LLMs providers for different steps in the pipeline for diversity and quality reasons. \n",
    "- You might need to set `OPENAI_API_KEY` in your environment variables to use the LLMs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from distilabel.llms import OpenAILLM\n",
    "\n",
    "llm = OpenAILLM(model=\"gpt-4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Synthesizing Generations\n",
    "\n",
    "We will now generate synthetic data using the LLM. We will use different prompt templates that were introduced and evaluated in various research papers. We call these `Tasks` within `distilabel`. Practically, there are several things to consider.\n",
    "\n",
    "- `Tasks` are and are not exhaustive. You can create your own `Tasks` based on your use-case.\n",
    "- `Tasks` are merely based on research papers and not always exact reproductions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate responses with `TextGeneration`\n",
    "\n",
    "We don't always need to use complex verified prompts. We can also just go for basic chat completion. Practically, there are several things to consider.\n",
    "\n",
    "- You might need to do some pre or post-processing to enure the data is formatted correctly.\n",
    "- If working with chat data, you might want to use the `ChatGeneration` task instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:04:18] </span><span style=\"color: #800000; text-decoration-color: #800000\">WARNING </span> WARNING:distilabel.step.text_generation:Step <span style=\"color: #008000; text-decoration-color: #008000\">'text_generation'</span> hasn't      <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/base.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">base.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/base.py#166\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">166</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         received a pipeline, and it hasn't been created within a `Pipeline`        <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">           </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         context. Please, use `with <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Pipeline</span><span style=\"font-weight: bold\">()</span> as pipeline:` and create the step    <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">           </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         within the context.                                                        <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">           </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:04:18]\u001b[0m\u001b[2;36m \u001b[0m\u001b[31mWARNING \u001b[0m WARNING:distilabel.step.text_generation:Step \u001b[32m'text_generation'\u001b[0m hasn't      \u001b]8;id=298338;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/base.py\u001b\\\u001b[2mbase.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=87738;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/base.py#166\u001b\\\u001b[2m166\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         received a pipeline, and it hasn't been created within a `Pipeline`        \u001b[2m           \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         context. Please, use `with \u001b[1;35mPipeline\u001b[0m\u001b[1m(\u001b[0m\u001b[1m)\u001b[0m as pipeline:` and create the step    \u001b[2m           \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         within the context.                                                        \u001b[2m           \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from distilabel.steps.tasks import TextGeneration\n",
    " \n",
    "text_generation = TextGeneration(name=\"text_generation\", llm=llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:04:22] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> 📝 Pipeline data will be written to         <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">128</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'/Users/davidberenstein/.cache/distilabel/pipelines/text_generation/44931</span> <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">5ac4554d4f71ad10f58a479a97f2bb1ff15/data'</span>                                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:04:22]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m 📝 Pipeline data will be written to         \u001b]8;id=173904;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=914129;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\u001b\\\u001b[2m128\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'/Users/davidberenstein/.cache/distilabel/pipelines/text_generation/44931\u001b[0m \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m5ac4554d4f71ad10f58a479a97f2bb1ff15/data'\u001b[0m                                 \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Waiting for all the steps to load<span style=\"color: #808000; text-decoration-color: #808000\">...</span>     <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">469</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Waiting for all the steps to load\u001b[33m...\u001b[0m     \u001b]8;id=125336;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=886245;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\u001b\\\u001b[2m469\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:04:25] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Steps loaded: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>                        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">483</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:04:25]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Steps loaded: \u001b[1;36m2\u001b[0m/\u001b[1;36m3\u001b[0m                        \u001b]8;id=753662;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=270176;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\u001b\\\u001b[2m483\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:04:27] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Steps loaded: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>                        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">483</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:04:27]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Steps loaded: \u001b[1;36m3\u001b[0m/\u001b[1;36m3\u001b[0m                        \u001b]8;id=76450;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=725549;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\u001b\\\u001b[2m483\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ✅ All the steps have been loaded!          <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">487</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ✅ All the steps have been loaded!          \u001b]8;id=719144;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=390237;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\u001b\\\u001b[2m487\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:04:28] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_dataset'</span><span style=\"font-weight: bold\">]</span> 🧬 Starting yielding batches from        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">911</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         generator step <span style=\"color: #008000; text-decoration-color: #008000\">'load_dataset'</span>. Offset: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>                                  <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:04:28]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_dataset'\u001b[0m\u001b[1m]\u001b[0m 🧬 Starting yielding batches from        \u001b]8;id=924272;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=156705;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\u001b\\\u001b[2m911\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         generator step \u001b[32m'load_dataset'\u001b[0m. Offset: \u001b[1;36m0\u001b[0m                                  \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_dataset'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'load_dataset'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_dataset'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'load_dataset'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=558521;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=441089;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/base.py:162: UserWarning: `use_system_prompt` is set to `True`, but no `system_prompt` in input batch, so it will be ignored.\n",
      "  return [self.format_input(input) for input in inputs]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_dataset'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'load_dataset'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_dataset'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'load_dataset'\u001b[0m  \u001b]8;id=976300;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=815559;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.text_generation'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in              <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'text_generation'</span>                                                         <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.text_generation'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in              \u001b]8;id=309716;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=866784;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'text_generation'\u001b[0m                                                         \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:04:36] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.text_generation'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'text_generation'</span> sending     <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> to output queue                                                   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:04:36]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.text_generation'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'text_generation'\u001b[0m sending     \u001b]8;id=92923;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=286436;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         batch \u001b[1;36m0\u001b[0m to output queue                                                   \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.text_generation'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step              <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'text_generation'</span>                                                         <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.text_generation'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step              \u001b]8;id=157645;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=460200;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'text_generation'\u001b[0m                                                         \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=755081;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=184859;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'keep_columns'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=191742;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=248246;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=510501;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=471952;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fa8ebad0a0cd439383067214aa6aa238",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from distilabel.pipeline import Pipeline\n",
    "from distilabel.steps import LoadHubDataset, KeepColumns\n",
    "from distilabel.steps.tasks import TextGeneration\n",
    "\n",
    "with Pipeline(name=\"text_generation\") as pipeline_text_generation:\n",
    "    load_hub_dataset = LoadHubDataset(\n",
    "        name=\"load_dataset\",\n",
    "        repo_id=\"DIBT/10k_prompts_ranked\",\n",
    "        output_mappings={\"prompt\": \"instruction\"},\n",
    "        split=\"train\",\n",
    "        num_examples=1\n",
    "    )\n",
    "    text_generation = TextGeneration(name=\"text_generation\", llm=llm)\n",
    "    keep_columns = KeepColumns(\n",
    "        name=\"keep_columns\",\n",
    "        columns=[\n",
    "            \"instruction\",\n",
    "            \"generation\",\n",
    "        ],\n",
    "    )\n",
    "    load_hub_dataset >> text_generation >> keep_columns\n",
    "    \n",
    "distiset_text_generation = pipeline_text_generation.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'instruction': 'Provide step-by-step instructions on how to make a safe and effective homemade all-purpose cleaner from common household ingredients. The guide should include measurements, tips for storing the cleaner, and additional variations or scents that can be added. Additionally, the guide should be written in clear and concise language, with helpful visuals or photographs to aid in the process.',\n",
       " 'generation': \"Here's a guide to making a homemade all-purpose cleaner. This guide doesn't include visuals or photographs as those can't be provided on this platform, but the steps are written clearly to guide you through the process effectively.\\n\\nIngredients:\\n- 1 cup of white vinegar\\n- 1 cup of water\\n- 1/2 lemon juice (optional for extra disinfecting power and fresh scent)\\n- 20-30 drops of essential oil such as lavender, tea tree, peppermint, or eucalyptus (optional for a pleasant aroma)\\n\\nSupplies:\\n- A measuring cup\\n- A spray bottle\\n- A funnel\"}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distiset_text_generation[\"default\"][\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Improve prompts with `SelfInstruct`\n",
    "\n",
    "Based on the paper [Self-Instruct: Aligning LM with Self Generated Instructions](https://arxiv.org/abs/2212.10560). It relies on rewriting an instruction based on certain critaria that are deemed important. Practically, there are several things to consider.\n",
    "\n",
    "- You might customize `criteria_for_query_generation` to improve the quality of the prompts to you domain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"# Task Description\\nDevelop  user queries that can be received by the given AI application and applicable to the provided context. Emphasize diversity in verbs and linguistic structures within the model's textual capabilities.\\n\\n# Criteria for Queries\\n\\nWrite each query on a separate line and avoid using numbered lists or bullet points.\\n\\n# AI Application\\n\\n\\n# Context\\n\\n\\n# Output\\n\""
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "### COMMENTED OUT BECAUSE IT REQUIRES RELOADING THE NOTEBOOK AND LLM ###\n",
    "# from distilabel.steps.tasks import SelfInstruct\n",
    " \n",
    "# self_instruct = SelfInstruct(name=\"self_instruct\", llm=llm, num_instructions=1)\n",
    "# self_instruct.load()\n",
    "# self_instruct._template.render()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from distilabel.pipeline import Pipeline\n",
    "from distilabel.steps import LoadHubDataset, KeepColumns\n",
    "from distilabel.steps.tasks import SelfInstruct\n",
    "\n",
    "with Pipeline(name=\"self_instruct\") as pipeline_self_instruct:\n",
    "    load_hub_dataset = LoadHubDataset(\n",
    "        name=\"load_dataset\",\n",
    "        repo_id=\"DIBT/10k_prompts_ranked\",\n",
    "        output_mappings={\"prompt\": \"input\"},\n",
    "        split=\"train\",\n",
    "        num_examples=1\n",
    "    )\n",
    "    self_instruct = SelfInstruct(name=\"self_instruct\", llm=llm, num_instructions=1)\n",
    "    keep_columns = KeepColumns(\n",
    "        name=\"keep_columns\",\n",
    "        columns=[\n",
    "            \"input\",\n",
    "            \"instructions\",\n",
    "        ],\n",
    "    )\n",
    "    load_hub_dataset >> self_instruct >> keep_columns\n",
    "    \n",
    "distiset_self_instruct = pipeline_self_instruct.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input': 'Provide step-by-step instructions on how to make a safe and effective homemade all-purpose cleaner from common household ingredients. The guide should include measurements, tips for storing the cleaner, and additional variations or scents that can be added. Additionally, the guide should be written in clear and concise language, with helpful visuals or photographs to aid in the process.',\n",
       " 'instructions': ['How can I create a safe and effective homemade all-purpose cleaner with common household ingredients?']}"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distiset_self_instruct[\"default\"][\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Improve responses with `EvolQuality`\n",
    "\n",
    "Based on the paper [WizardLM: Empowering Large Language Models to Follow Complex Instructions](https://arxiv.org/pdf/2312.15685). It relies on an LLM to improve the quality response given an input based on different criteria. Practically, there are several things to consider.\n",
    "\n",
    "- You might want to directly link this pipeline with the `TextGeneration` step.\n",
    "- You might want to generate with `num_evolutions>1` so we directly have more than two options for preference data annotation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:38:27] </span><span style=\"color: #800000; text-decoration-color: #800000\">WARNING </span> WARNING:distilabel.step.evol_quality:Step <span style=\"color: #008000; text-decoration-color: #008000\">'evol_quality'</span> hasn't received a <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/base.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">base.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/base.py#166\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">166</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         pipeline, and it hasn't been created within a `Pipeline` context. Please,  <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">           </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         use `with <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Pipeline</span><span style=\"font-weight: bold\">()</span> as pipeline:` and create the step within the context. <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">           </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:38:27]\u001b[0m\u001b[2;36m \u001b[0m\u001b[31mWARNING \u001b[0m WARNING:distilabel.step.evol_quality:Step \u001b[32m'evol_quality'\u001b[0m hasn't received a \u001b]8;id=55601;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/base.py\u001b\\\u001b[2mbase.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=810939;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/base.py#166\u001b\\\u001b[2m166\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         pipeline, and it hasn't been created within a `Pipeline` context. Please,  \u001b[2m           \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         use `with \u001b[1;35mPipeline\u001b[0m\u001b[1m(\u001b[0m\u001b[1m)\u001b[0m as pipeline:` and create the step within the context. \u001b[2m           \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "{'HELPFULNESS': \"I want you to act as a Response Rewriter.\\nYour goal is to enhance the quality of the response given by an AI assistant to the #Given Prompt# through rewriting.\\nBut the rewritten prompt must be reasonable and must be understood and responded by humans.\\nYour rewriting cannot omit the non-text parts such as the table and code in #Given Prompt# and #Given Response#. Also, please do not omit the input in #Given Prompt#.\\n\\nYou Should enhance the quality of the response using the following method: \\nPlease make the Response more helpful to the user.\\nYou should try your best not to make the #Rewritten Response# become verbose, #Rewritten Response# can only add 10 to 20 words into #Given Response#.\\n'#Given Response#', '#Rewritten Response#', 'given response' and 'rewritten response' are not allowed to appear in #Rewritten Response#\\n#Given Prompt#:\\n<PROMPT>\\n#Given Response#:\\n<RESPONSE>\\n#Rewritten Response#:\\n\",\n",
       " 'RELEVANCE': \"I want you to act as a Response Rewriter.\\nYour goal is to enhance the quality of the response given by an AI assistant to the #Given Prompt# through rewriting.\\nBut the rewritten prompt must be reasonable and must be understood and responded by humans.\\nYour rewriting cannot omit the non-text parts such as the table and code in #Given Prompt# and #Given Response#. Also, please do not omit the input in #Given Prompt#.\\n\\nYou Should enhance the quality of the response using the following method: \\nPlease make the Response more relevant to #Given Prompt#.\\nYou should try your best not to make the #Rewritten Response# become verbose, #Rewritten Response# can only add 10 to 20 words into #Given Response#.\\n'#Given Response#', '#Rewritten Response#', 'given response' and 'rewritten response' are not allowed to appear in #Rewritten Response#\\n#Given Prompt#:\\n<PROMPT>\\n#Given Response#:\\n<RESPONSE>\\n#Rewritten Response#:\\n\",\n",
       " 'DEEPENING': \"I want you to act as a Response Rewriter.\\nYour goal is to enhance the quality of the response given by an AI assistant to the #Given Prompt# through rewriting.\\nBut the rewritten prompt must be reasonable and must be understood and responded by humans.\\nYour rewriting cannot omit the non-text parts such as the table and code in #Given Prompt# and #Given Response#. Also, please do not omit the input in #Given Prompt#.\\n\\nYou Should enhance the quality of the response using the following method: \\nPlease make the Response more in-depth.\\nYou should try your best not to make the #Rewritten Response# become verbose, #Rewritten Response# can only add 10 to 20 words into #Given Response#.\\n'#Given Response#', '#Rewritten Response#', 'given response' and 'rewritten response' are not allowed to appear in #Rewritten Response#\\n#Given Prompt#:\\n<PROMPT>\\n#Given Response#:\\n<RESPONSE>\\n#Rewritten Response#:\\n\",\n",
       " 'CREATIVITY': \"I want you to act as a Response Rewriter.\\nYour goal is to enhance the quality of the response given by an AI assistant to the #Given Prompt# through rewriting.\\nBut the rewritten prompt must be reasonable and must be understood and responded by humans.\\nYour rewriting cannot omit the non-text parts such as the table and code in #Given Prompt# and #Given Response#. Also, please do not omit the input in #Given Prompt#.\\n\\nYou Should enhance the quality of the response using the following method: \\nPlease increase the creativity of the response.\\nYou should try your best not to make the #Rewritten Response# become verbose, #Rewritten Response# can only add 10 to 20 words into #Given Response#.\\n'#Given Response#', '#Rewritten Response#', 'given response' and 'rewritten response' are not allowed to appear in #Rewritten Response#\\n#Given Prompt#:\\n<PROMPT>\\n#Given Response#:\\n<RESPONSE>\\n#Rewritten Response#:\\n\",\n",
       " 'DETAILS': \"I want you to act as a Response Rewriter.\\nYour goal is to enhance the quality of the response given by an AI assistant to the #Given Prompt# through rewriting.\\nBut the rewritten prompt must be reasonable and must be understood and responded by humans.\\nYour rewriting cannot omit the non-text parts such as the table and code in #Given Prompt# and #Given Response#. Also, please do not omit the input in #Given Prompt#.\\n\\nYou Should enhance the quality of the response using the following method: \\nPlease increase the detail level of Response.\\nYou should try your best not to make the #Rewritten Response# become verbose, #Rewritten Response# can only add 10 to 20 words into #Given Response#.\\n'#Given Response#', '#Rewritten Response#', 'given response' and 'rewritten response' are not allowed to appear in #Rewritten Response#\\n#Given Prompt#:\\n<PROMPT>\\n#Given Response#:\\n<RESPONSE>\\n#Rewritten Response#:\\n\"}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from distilabel.steps.tasks import EvolQuality\n",
    "\n",
    "evol_quality = EvolQuality(name=\"evol_quality\", llm=llm, num_evolutions=1)\n",
    "evol_quality.mutation_templates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:39:51] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> 📝 Pipeline data will be written to         <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">128</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'/Users/davidberenstein/.cache/distilabel/pipelines/evol_quality/08e62411</span> <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">01b70f873be5b07150639b18f384c321/data'</span>                                    <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:39:51]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m 📝 Pipeline data will be written to         \u001b]8;id=647602;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=434657;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\u001b\\\u001b[2m128\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'/Users/davidberenstein/.cache/distilabel/pipelines/evol_quality/08e62411\u001b[0m \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m01b70f873be5b07150639b18f384c321/data'\u001b[0m                                    \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Waiting for all the steps to load<span style=\"color: #808000; text-decoration-color: #808000\">...</span>     <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">469</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Waiting for all the steps to load\u001b[33m...\u001b[0m     \u001b]8;id=589997;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=680405;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\u001b\\\u001b[2m469\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:39:54] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Steps loaded: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>                        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">483</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:39:54]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Steps loaded: \u001b[1;36m3\u001b[0m/\u001b[1;36m3\u001b[0m                        \u001b]8;id=894616;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=832038;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\u001b\\\u001b[2m483\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ✅ All the steps have been loaded!          <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">487</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ✅ All the steps have been loaded!          \u001b]8;id=321694;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=259194;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\u001b\\\u001b[2m487\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:39:55] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 🧬 Starting yielding batches from generator <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">911</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span>. Offset: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>                                               <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:39:55]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 🧬 Starting yielding batches from generator \u001b]8;id=919329;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=741968;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\u001b\\\u001b[2m911\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         step \u001b[32m'load_data'\u001b[0m. Offset: \u001b[1;36m0\u001b[0m                                               \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> to      <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         output queue                                                              <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'load_data'\u001b[0m sending batch \u001b[1;36m0\u001b[0m to      \u001b]8;id=739560;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=546843;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         output queue                                                              \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span>        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'load_data'\u001b[0m        \u001b]8;id=661725;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=734052;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in <span style=\"color: #008000; text-decoration-color: #008000\">'evol_quality'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in \u001b[32m'evol_quality'\u001b[0m  \u001b]8;id=252301;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=966069;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:39:56] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 🔄 Ran iteration <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> evolving <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span> responses!  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">base.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py#197\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">197</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:39:56]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 🔄 Ran iteration \u001b[1;36m0\u001b[0m evolving \u001b[1;36m1\u001b[0m responses!  \u001b]8;id=758748;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py\u001b\\\u001b[2mbase.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=767744;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py#197\u001b\\\u001b[2m197\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 14:39:58] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 🔄 Ran iteration <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span> evolving <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span> responses!  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">base.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py#197\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">197</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 14:39:58]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 🔄 Ran iteration \u001b[1;36m1\u001b[0m evolving \u001b[1;36m1\u001b[0m responses!  \u001b]8;id=626143;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py\u001b\\\u001b[2mbase.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=8234;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py#197\u001b\\\u001b[2m197\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'evol_quality'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'evol_quality'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=21500;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=854560;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'evol_quality'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'evol_quality'\u001b[0m  \u001b]8;id=174590;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=645606;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=91255;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=246220;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'keep_columns'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=323613;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=750524;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=579760;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=75568;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6071c74bdce741e1aea56a63bf2e0ee3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from distilabel.pipeline import Pipeline\n",
    "from distilabel.steps import LoadDataFromDicts, KeepColumns\n",
    "from distilabel.steps.tasks import EvolQuality\n",
    "\n",
    "with Pipeline(name=\"evol_quality\") as pipeline_evol_quality:\n",
    "    load_data = LoadDataFromDicts(\n",
    "        name=\"load_data\",\n",
    "        data=[\n",
    "            {\n",
    "                'instruction': \"What did Leonarda Da Vinci focus on during his life?\",\n",
    "                'response': \"He was an Italian scientist and engineer in the renaissance.\",\n",
    "            }\n",
    "        ],\n",
    "    )\n",
    "    evol_quality = EvolQuality(name=\"evol_quality\", llm=llm, num_evolutions=1)\n",
    "    keep_columns = KeepColumns(\n",
    "        name=\"keep_columns\",\n",
    "        columns=[\n",
    "            \"instruction\",\n",
    "            \"response\",\n",
    "            \"evolved_response\",\n",
    "        ],\n",
    "    )\n",
    "    load_data >> evol_quality >> keep_columns\n",
    "\n",
    "distiset_evol_quality = pipeline_evol_quality.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'instruction': 'What did Leonarda Da Vinci focus on during his life?',\n",
       " 'response': 'He was an Italian scientist and engineer in the renaissance.',\n",
       " 'evolved_response': 'Leonardo Da Vinci had a multifaceted focus throughout his life, which spanned interests in art, science, engineering, and the natural world.'}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distiset_evol_quality[\"default\"][\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Synthesizing AI Feedback with LLMs as Judges \n",
    "\n",
    "We will now generate synthetic evaluations using the LLM. This once again relies on prompt templates and `Tasks` as with synthesizing generations. Practically, there are several things to consider.\n",
    "\n",
    "- Only several LLMs can actually generate evaluations that align with human evaluations. Higher-end propietary models from companies like `mistral` and `OpenAI` are better at this. For open-source models, you might want to use `Prometheus 2.0`.\n",
    "- You can use evaluate based on differnt aspects like helpfulness, relevance, and fluency, however, given the cost, an overall rating is usually sufficient."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Absolute evaluation of reponses using `UltraFeedback`\n",
    "\n",
    "Based on the paper [UltraFeedback: Boosting Language Models with High-quality Feedback](https://arxiv.org/abs/2310.01377). In this case, we will generate an absolute feedback score based on an overall rating. Practically, there are several things to consider.\n",
    "\n",
    "- A single overall rating is usually sufficient but UltraFeedback also covers (Instruction-following, truthfullness, honesty, helpfulness).\n",
    "- `PrometheusEval` and `Prometheus 2.0` are a good alternative for open-source models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Your role is to evaluate text quality based on given criteria.\\nYou\\'ll receive an instructional description (\"Instruction\") and {no_texts} text outputs (\"Text\").\\nUnderstand and interpret instructions to evaluate effectively.\\nProvide annotations for each text with a rating and rationale.\\nThe {no_texts} texts given are independent, and should be evaluated separately.\\n'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from distilabel.steps.tasks import UltraFeedback\n",
    "\n",
    "ultra_feedback = UltraFeedback(name=\"ultra_feedback\", llm=llm, aspect=\"overall-rating\")\n",
    "ultra_feedback._system_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 15:03:15] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> 📝 Pipeline data will be written to         <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">128</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'/Users/davidberenstein/.cache/distilabel/pipelines/ultra_feedback/a6fdea</span> <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">ccbcf7ffeaa4452f2c53ece2658648c5a3/data'</span>                                  <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 15:03:15]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m 📝 Pipeline data will be written to         \u001b]8;id=525058;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=332371;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\u001b\\\u001b[2m128\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'/Users/davidberenstein/.cache/distilabel/pipelines/ultra_feedback/a6fdea\u001b[0m \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32mccbcf7ffeaa4452f2c53ece2658648c5a3/data'\u001b[0m                                  \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Waiting for all the steps to load<span style=\"color: #808000; text-decoration-color: #808000\">...</span>     <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">469</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Waiting for all the steps to load\u001b[33m...\u001b[0m     \u001b]8;id=481595;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=203951;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\u001b\\\u001b[2m469\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 15:03:18] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Steps loaded: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>                        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">483</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 15:03:18]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Steps loaded: \u001b[1;36m3\u001b[0m/\u001b[1;36m3\u001b[0m                        \u001b]8;id=818933;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=386350;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\u001b\\\u001b[2m483\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ✅ All the steps have been loaded!          <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">487</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ✅ All the steps have been loaded!          \u001b]8;id=361139;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=202657;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\u001b\\\u001b[2m487\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 15:03:19] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 🧬 Starting yielding batches from generator <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">911</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span>. Offset: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>                                               <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 15:03:19]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 🧬 Starting yielding batches from generator \u001b]8;id=457624;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=487822;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\u001b\\\u001b[2m911\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         step \u001b[32m'load_data'\u001b[0m. Offset: \u001b[1;36m0\u001b[0m                                               \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> to      <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         output queue                                                              <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'load_data'\u001b[0m sending batch \u001b[1;36m0\u001b[0m to      \u001b]8;id=72269;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=157062;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         output queue                                                              \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span>        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'load_data'\u001b[0m        \u001b]8;id=184314;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=666648;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.ultra_feedback'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in               <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'ultra_feedback'</span>                                                          <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.ultra_feedback'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in               \u001b]8;id=997542;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=820887;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'ultra_feedback'\u001b[0m                                                          \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 15:03:29] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.ultra_feedback'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'ultra_feedback'</span> sending batch <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> to output queue                                                         <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 15:03:29]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.ultra_feedback'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'ultra_feedback'\u001b[0m sending batch \u001b]8;id=212344;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=775963;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[1;36m0\u001b[0m to output queue                                                         \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.ultra_feedback'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step               <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'ultra_feedback'</span>                                                          <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.ultra_feedback'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step               \u001b]8;id=373032;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=773587;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'ultra_feedback'\u001b[0m                                                          \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=210480;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=166709;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'keep_columns'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=475121;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=174751;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=743364;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=120804;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "54b7c88922474cd088ef1de361157716",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from distilabel.pipeline import Pipeline\n",
    "from distilabel.steps import LoadDataFromDicts, KeepColumns\n",
    "from distilabel.steps.tasks import UltraFeedback\n",
    "\n",
    "with Pipeline(name=\"ultra_feedback\") as pipeline_ultra_feedback:\n",
    "    load_data = LoadDataFromDicts(\n",
    "        name=\"load_data\",\n",
    "        data=[\n",
    "            {\n",
    "                'instruction': \"What did Leonarda Da Vinci focus on during his life?\",\n",
    "                'generations': [\n",
    "                    \"He was an Italian scientist and engineer in the renaissance.\",\n",
    "                    \"He was a painter, sculptor, architect, and engineer.\",\n",
    "                ]\n",
    "            }\n",
    "        ],\n",
    "    )\n",
    "    ultra_feedback = UltraFeedback(name=\"ultra_feedback\", llm=llm, aspect=\"overall-rating\")\n",
    "    keep_columns = KeepColumns(\n",
    "        name=\"keep_columns\",\n",
    "        columns=[\n",
    "            \"instruction\",\n",
    "            \"generations\",\n",
    "            \"ratings\",\n",
    "            \"rationales\"\n",
    "        ],\n",
    "    )\n",
    "    load_data >> ultra_feedback >> keep_columns\n",
    "    \n",
    "distiset_ultra_feedback = pipeline_ultra_feedback.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'instruction': 'What did Leonarda Da Vinci focus on during his life?',\n",
       " 'generations': ['He was an Italian scientist and engineer in the renaissance.',\n",
       "  'He was a painter, sculptor, architect, and engineer.'],\n",
       " 'ratings': [3, 4],\n",
       " 'rationales': ['The text correctly identifies Leonardo Da Vinci as a scientist and engineer during the renaissance, which is partially accurate. However, it fails to include essential aspects like his focus on painting, sculpture, and architecture, which is significant misinformation. Hence it partially follows the instruction.',\n",
       "  \"This text gives a more comprehensive answer, touching on Leonardo Da Vinci's roles as a painter, sculptor, architect, and engineer. However, it could be improved with a note on his contributions to science and inventiveness,\"]}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distiset_ultra_feedback[\"default\"][\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Relative evaluation (ranking) of responses using ranking `QualityScorer`\n",
    "\n",
    "Based on the paper [WizardLM: Empowering Large Language Models to Follow Complex Instructions](https://arxiv.org/pdf/2312.15685). Practically, there are several things to consider.\n",
    "\n",
    "- You might want use the `PairRM` model for predictive open source alternative to LLMs as relative judges.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Rank the following pair of instructions and responses according to their quality. Your evaluation should consider factors such as helpfulness, relevance, accuracy, depth, creativity, and level of detail of the response. Score 1-0.\\nScore each response from 1 to 0, with 1 reserved for responses that are already very well written and cannot be improved further. You should respond with the format:\\n[1] Score: 1\\n[2] Score: 2\\n...\\n#Question#: \\n#Response List#:\\n'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "### COMMENTED OUT BECAUSE IT REQUIRES RELOADING THE NOTEBOOK AND LLM ###\n",
    "# from distilabel.steps.tasks import QualityScorer\n",
    "\n",
    "# quality_scorer = QualityScorer(name=\"quality_scorer\", llm=llm)\n",
    "# quality_scorer.load()\n",
    "# quality_scorer._template.render()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 15:39:42] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> 📝 Pipeline data will be written to         <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">128</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'/Users/davidberenstein/.cache/distilabel/pipelines/quality_scorer/a70acf</span> <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">76907c8012403c03e551bac21479f726a2/data'</span>                                  <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 15:39:42]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m 📝 Pipeline data will be written to         \u001b]8;id=525333;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=11003;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\u001b\\\u001b[2m128\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'/Users/davidberenstein/.cache/distilabel/pipelines/quality_scorer/a70acf\u001b[0m \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m76907c8012403c03e551bac21479f726a2/data'\u001b[0m                                  \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Waiting for all the steps to load<span style=\"color: #808000; text-decoration-color: #808000\">...</span>     <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">469</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Waiting for all the steps to load\u001b[33m...\u001b[0m     \u001b]8;id=171884;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=482659;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\u001b\\\u001b[2m469\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 15:39:44] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Steps loaded: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>                        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">483</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 15:39:44]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Steps loaded: \u001b[1;36m3\u001b[0m/\u001b[1;36m3\u001b[0m                        \u001b]8;id=432404;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=849789;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\u001b\\\u001b[2m483\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ✅ All the steps have been loaded!          <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">487</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ✅ All the steps have been loaded!          \u001b]8;id=342759;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=806367;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\u001b\\\u001b[2m487\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 15:39:45] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 🧬 Starting yielding batches from generator <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">911</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span>. Offset: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>                                               <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 15:39:45]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 🧬 Starting yielding batches from generator \u001b]8;id=662106;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=153908;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\u001b\\\u001b[2m911\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         step \u001b[32m'load_data'\u001b[0m. Offset: \u001b[1;36m0\u001b[0m                                               \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> to      <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         output queue                                                              <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'load_data'\u001b[0m sending batch \u001b[1;36m0\u001b[0m to      \u001b]8;id=28396;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=114553;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         output queue                                                              \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_data'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'load_data'</span>        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_data'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'load_data'\u001b[0m        \u001b]8;id=30709;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=415940;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.quality_scorer'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in               <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'quality_scorer'</span>                                                          <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.quality_scorer'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in               \u001b]8;id=265795;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=862639;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'quality_scorer'\u001b[0m                                                          \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 15:39:48] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.quality_scorer'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'quality_scorer'</span> sending batch <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> to output queue                                                         <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 15:39:48]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.quality_scorer'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'quality_scorer'\u001b[0m sending batch \u001b]8;id=385639;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=333455;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[1;36m0\u001b[0m to output queue                                                         \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.quality_scorer'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step               <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'quality_scorer'</span>                                                          <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.quality_scorer'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step               \u001b]8;id=642072;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=987302;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'quality_scorer'\u001b[0m                                                          \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=967199;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=206809;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'keep_columns'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=295292;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=575483;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=816051;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=69966;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fbfdb04fa2d1403b998a2a0988d70f74",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from distilabel.pipeline import Pipeline\n",
    "from distilabel.steps import LoadDataFromDicts, KeepColumns\n",
    "from distilabel.steps.tasks import QualityScorer\n",
    "\n",
    "with Pipeline(name=\"quality_scorer\") as pipeline_quality_scorer:\n",
    "    load_data = LoadDataFromDicts(\n",
    "        name=\"load_data\",\n",
    "        data=[\n",
    "            {\n",
    "                'instruction': \"What did Leonarda Da Vinci focus on during his life?\",\n",
    "                'responses': [\n",
    "                    \"He was an Italian scientist and engineer in the renaissance.\",\n",
    "                    \"He was a painter, sculptor, architect, and engineer.\",\n",
    "                ]\n",
    "            }\n",
    "        ],\n",
    "    )\n",
    "    quality_scorer = QualityScorer(name=\"quality_scorer\", llm=llm)\n",
    "    keep_columns = KeepColumns(\n",
    "        name=\"keep_columns\",\n",
    "        columns=[\n",
    "            \"instruction\",\n",
    "            \"responses\",\n",
    "            \"scores\",\n",
    "        ],\n",
    "    )\n",
    "    load_data >> quality_scorer >> keep_columns\n",
    "\n",
    "distiset_quality_scorer = pipeline_quality_scorer.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'instruction': 'What did Leonarda Da Vinci focus on during his life?',\n",
       " 'responses': ['He was an Italian scientist and engineer in the renaissance.',\n",
       "  'He was a painter, sculptor, architect, and engineer.'],\n",
       " 'scores': [1.0, 2.0]}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distiset_quality_scorer[\"default\"][\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A full pipeline \n",
    "\n",
    "We will now combine all the steps into a full pipeline. Practically, there are several things to consider.\n",
    "\n",
    "- You can be as creative as you want with the pipeline.\n",
    "- More complex pipelines generally require more computational resources, results are cached but it can still be expensive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 17:06:17] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> 📝 Pipeline data will be written to         <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">128</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'/Users/davidberenstein/.cache/distilabel/pipelines/text_generation/f0899</span> <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">6f04836de25629e1873a295576ef48f8511/data'</span>                                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 17:06:17]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m 📝 Pipeline data will be written to         \u001b]8;id=35012;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=313736;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#128\u001b\\\u001b[2m128\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'/Users/davidberenstein/.cache/distilabel/pipelines/text_generation/f0899\u001b[0m \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m6f04836de25629e1873a295576ef48f8511/data'\u001b[0m                                 \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Waiting for all the steps to load<span style=\"color: #808000; text-decoration-color: #808000\">...</span>     <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">469</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Waiting for all the steps to load\u001b[33m...\u001b[0m     \u001b]8;id=215993;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=853213;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#469\u001b\\\u001b[2m469\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 17:06:20] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Steps loaded: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5</span>                        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">483</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 17:06:20]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Steps loaded: \u001b[1;36m4\u001b[0m/\u001b[1;36m5\u001b[0m                        \u001b]8;id=84326;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=614509;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\u001b\\\u001b[2m483\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 17:06:22] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ⏳ Steps loaded: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5</span>                        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">483</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 17:06:22]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ⏳ Steps loaded: \u001b[1;36m5\u001b[0m/\u001b[1;36m5\u001b[0m                        \u001b]8;id=883449;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=703753;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#483\u001b\\\u001b[2m483\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.pipeline.local'</span><span style=\"font-weight: bold\">]</span> ✅ All the steps have been loaded!          <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">487</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.pipeline.local'\u001b[0m\u001b[1m]\u001b[0m ✅ All the steps have been loaded!          \u001b]8;id=981867;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=944977;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#487\u001b\\\u001b[2m487\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 17:06:23] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_dataset'</span><span style=\"font-weight: bold\">]</span> 🧬 Starting yielding batches from        <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">911</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         generator step <span style=\"color: #008000; text-decoration-color: #008000\">'load_dataset'</span>. Offset: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>                                  <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 17:06:23]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_dataset'\u001b[0m\u001b[1m]\u001b[0m 🧬 Starting yielding batches from        \u001b]8;id=335966;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=506991;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#911\u001b\\\u001b[2m911\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         generator step \u001b[32m'load_dataset'\u001b[0m. Offset: \u001b[1;36m0\u001b[0m                                  \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/base.py:162: UserWarning: `use_system_prompt` is set to `True`, but no `system_prompt` in input batch, so it will be ignored.\n",
      "  return [self.format_input(input) for input in inputs]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_dataset'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'load_dataset'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_dataset'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'load_dataset'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=661724;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=594605;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.load_dataset'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'load_dataset'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.load_dataset'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'load_dataset'\u001b[0m  \u001b]8;id=297580;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=374297;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.text_generation'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in              <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'text_generation'</span>                                                         <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.text_generation'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in              \u001b]8;id=943529;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=286019;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'text_generation'\u001b[0m                                                         \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 17:06:34] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.text_generation'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'text_generation'</span> sending     <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> to output queue                                                   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 17:06:34]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.text_generation'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'text_generation'\u001b[0m sending     \u001b]8;id=283979;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=44328;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         batch \u001b[1;36m0\u001b[0m to output queue                                                   \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.text_generation'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step              <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'text_generation'</span>                                                         <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.text_generation'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step              \u001b]8;id=202262;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=208153;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'text_generation'\u001b[0m                                                         \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in <span style=\"color: #008000; text-decoration-color: #008000\">'evol_quality'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in \u001b[32m'evol_quality'\u001b[0m  \u001b]8;id=832011;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=470177;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 17:06:45] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 🔄 Ran iteration <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> evolving <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4</span> responses!  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">base.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py#197\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">197</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 17:06:45]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 🔄 Ran iteration \u001b[1;36m0\u001b[0m evolving \u001b[1;36m4\u001b[0m responses!  \u001b]8;id=394257;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py\u001b\\\u001b[2mbase.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=669293;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py#197\u001b\\\u001b[2m197\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 17:06:55] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 🔄 Ran iteration <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span> evolving <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4</span> responses!  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">base.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py#197\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">197</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 17:06:55]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 🔄 Ran iteration \u001b[1;36m1\u001b[0m evolving \u001b[1;36m4\u001b[0m responses!  \u001b]8;id=274839;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py\u001b\\\u001b[2mbase.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=499114;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/steps/tasks/evol_quality/base.py#197\u001b\\\u001b[2m197\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'evol_quality'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'evol_quality'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=510174;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=149534;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.evol_quality'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'evol_quality'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.evol_quality'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'evol_quality'\u001b[0m  \u001b]8;id=660688;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=377021;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.quality_scorer'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in               <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'quality_scorer'</span>                                                          <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.quality_scorer'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in               \u001b]8;id=686341;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=636779;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'quality_scorer'\u001b[0m                                                          \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[06/05/24 17:06:57] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.quality_scorer'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'quality_scorer'</span> sending batch <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> to output queue                                                         <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[06/05/24 17:06:57]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.quality_scorer'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'quality_scorer'\u001b[0m sending batch \u001b]8;id=182979;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=166491;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[1;36m0\u001b[0m to output queue                                                         \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.quality_scorer'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step               <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'quality_scorer'</span>                                                          <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.quality_scorer'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step               \u001b]8;id=744413;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=740060;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'quality_scorer'\u001b[0m                                                          \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📦 Processing batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span> in <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">960</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📦 Processing batch \u001b[1;36m0\u001b[0m in \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=984370;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=372994;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#960\u001b\\\u001b[2m960\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 📨 Step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span> sending batch <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>   <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">998</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         to output queue                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 📨 Step \u001b[32m'keep_columns'\u001b[0m sending batch \u001b[1;36m0\u001b[0m   \u001b]8;id=81532;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=937719;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#998\u001b\\\u001b[2m998\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         to output queue                                                           \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'distilabel.step.keep_columns'</span><span style=\"font-weight: bold\">]</span> 🏁 Finished running step <span style=\"color: #008000; text-decoration-color: #008000\">'keep_columns'</span>  <a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">local.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">880</span></a>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m                   \u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m \u001b[1m[\u001b[0m\u001b[32m'distilabel.step.keep_columns'\u001b[0m\u001b[1m]\u001b[0m 🏁 Finished running step \u001b[32m'keep_columns'\u001b[0m  \u001b]8;id=224577;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py\u001b\\\u001b[2mlocal.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=263467;file:///Users/davidberenstein/Documents/programming/argilla/data-for-fine-tuning-llms/.env/lib/python3.10/site-packages/distilabel/pipeline/local.py#880\u001b\\\u001b[2m880\u001b[0m\u001b]8;;\u001b\\\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cb4c15a7a2e04e6f8bff02d41e72e0b1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from distilabel.pipeline import Pipeline\n",
    "from distilabel.steps import LoadHubDataset, KeepColumns\n",
    "from distilabel.steps.tasks import TextGeneration, EvolQuality, QualityScorer\n",
    "\n",
    "with Pipeline(name=\"text_generation\") as pipeline_complete:\n",
    "    load_hub_dataset = LoadHubDataset(\n",
    "        name=\"load_dataset\",\n",
    "        repo_id=\"DIBT/10k_prompts_ranked\",\n",
    "        output_mappings={\"prompt\": \"instruction\"},\n",
    "        split=\"train\",\n",
    "        num_examples=4\n",
    "    )\n",
    "    text_generation = TextGeneration(name=\"text_generation\", llm=llm)\n",
    "    evol_quality = EvolQuality(name=\"evol_quality\", llm=llm, num_evolutions=2, store_evolutions=True, input_mappings={\"response\": \"generation\"})\n",
    "    quality_scorer = QualityScorer(name=\"quality_scorer\", llm=llm, input_mappings={\"responses\": \"evolved_responses\"})\n",
    "    keep_columns = KeepColumns(\n",
    "        name=\"keep_columns\",\n",
    "        columns=[\n",
    "            \"instruction\",\n",
    "            \"evolved_responses\",\n",
    "            \"scores\"\n",
    "        ],\n",
    "    )\n",
    "    load_hub_dataset >> text_generation >> evol_quality >> quality_scorer >> keep_columns\n",
    "    \n",
    "    \n",
    "distiset_complete = pipeline_complete.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'instruction': ['Provide step-by-step instructions on how to make a safe and effective homemade all-purpose cleaner from common household ingredients. The guide should include measurements, tips for storing the cleaner, and additional variations or scents that can be added. Additionally, the guide should be written in clear and concise language, with helpful visuals or photographs to aid in the process.',\n",
       "  'Write a personal essay of at least 1000 words discussing how embracing vulnerability and authenticity has affected your life. Use specific examples from your own experiences to support your arguments and make sure to address the following questions:',\n",
       "  'In this research, we aim to investigate how technology can moderate the correlation between knowledge management practices and the overall performance of an organization. This analysis will focus on specific technological tools and their impact on knowledge management in relation to various aspects of organizational performance. Additionally, we will explore how this relationship may vary across different industries and organizational structures.',\n",
       "  \"Did Karl Marx's theories on centralizing credit have anything to do with our current central banking system?\"],\n",
       " 'evolved_responses': [[\"Creating a homemade all-purpose cleaner is not only cost-effective, but also eco-friendly as it limits chemical pollutants in your home. Here's a list of simple ingredients and step-by-step instructions.\\n\\nEssentials:\\n1. White Vinegar\\n2. Baking Soda\\n3. Lemon or Essential Oils for aromatic variations\\n4. An Empty, Clean, Dry Spray Bottle\\n5. A Mixing Bowl\\n6. A Funnel (for smoother bottle filling)\\n7. Water\\n\\nStep 1: Pour a cup of white vinegar into your mixing bowl. It's as easy as that to begin. White vinegar is your cleaning agent here.\",\n",
       "   'Creating a homemade all-purpose cleaner is not only cost-effective, but also eco-friendly, aiding in reducing the chemical pollutants in your home. Below you will find a list of basic ingredients and a step-by-step guide to prepare it.\\n\\nNecessities:\\n1. White Vinegar\\n2. Baking Soda\\n3. Lemon or Essential Oils for aromatic variations\\n4. An Empty, Clean, Dry Spray Bottle\\n5. A Mixing Bowl\\n6. A Funnel (for effortless transition of mix into the bottle)\\n7. Water\\n\\nStep 1: Kickoff the process by pouring a simple cup of white vinegar into your mixing'],\n",
       "  ['How has exposing your layers of vulnerability transformed your character?  \\nHow do you view the connection between authenticity and showing vulnerability?  \\nCan you detail the challenges encountered on this journey, and the ways you addressed them?\\n\\nIn an era dominated by the impeccably curated representation of life on social media platforms and the digitally enhanced glamour of celebrity lifestyle, choosing to embrace and proclaim vulnerability and authenticity displays a sense of bravery. Like a resilient sail bracing against the fierce wind, my decision to follow the path of vulnerability and authenticity has significantly molded and influenced my life, by altering my values and consequently rewriting my narrative.\\n\\nInitially, penetrating the tough exterior of perceived',\n",
       "   'What changes have occurred to your character through revealing your vulnerabilities?  \\nWhat are your perspectives on the relationships between authenticity and vulnerability?  \\nCould you delve into the challenges you faced on this journey and how you tackled them?\\n\\nIn a time defined by perfectly presented lives on social media platforms and digitally enhanced glamour of celebrities, the decision to acknowledge and exhibit vulnerability and authenticity reflects courage. Like a sturdy sail riding against a potent wind, opting to navigate the course of vulnerability and authenticity has profoundly shaped my life by changing my values and consequently revising my life story.\\n\\nInitially, piercing through the tough facade of perceived'],\n",
       "  [\"1. Introduction\\n\\nGrasping the association between technology and knowledge management is pivotal for businesses aspiring to bolster their overall performance. The term 'Knowledge Management' is employed to delineate the practices an organization utilizes to identify, create, represent, and disseminate knowledge for reutilization, fostering awareness, and learning throughout the entity. Conversely, technology has the potential to modify these practices in numerous ways, from amplifying communication and collaboration to facilitating the capture and preservation of knowledge.\\n\\n2. Theoretical background\\n\\nA wealth of research has unearthed a positive correlation between proficient knowledge management and the enhancement of organizational performance. Technology serves as a considerable facilit\",\n",
       "   \"1. Introduction\\n\\nUnderstanding the nexus between technology and knowledge management is crucial for corporations seeking to significantly improve their overall operational performance. 'Knowledge Management' typically encompasses the mechanisms an organization adopts to recognize, develop, represent, and distribute knowledge for reuse, to promote consciousness and cultivate learning across the organization. Meanwhile, technology influences these practices in diverse ways, from strengthening communication and fostering collaboration to aiding the capture and conservation of knowledge.\\n\\n2. Theoretical background\\n\\nPlentiful research evidence points towards a positive link between effective knowledge management and the escalation of organizational productivity and efficiency. Technology acts as a significant facilitator in this process, potentially transforming the performance outcomes\"],\n",
       "  [\"Undoubtedly, Karl Marx's advocated for a unified banking system in his writings, but crediting our current system solely to his theories would be a stretch. Our present central banking system has been shaped over centuries, influenced by a multitude of economists, political choices, and historical events. \\n\\nIt's plausible that certain elements of Marx’s standpoints on centralized credit have affected economic theories and perhaps indirectly molded certain facets of our current banking system. However, most of today's central banks function within a framework rooted in capitalist values, which Marx was intrinsically against.\\n\\nMarx perceived central banks as a means for state control over capital, to foster\",\n",
       "   \"Indeed, Karl Marx's theories did advocate for a centralized banking system. However, attributing the entirety of our current system to his concepts alone is an oversimplification. The system in place today is the result of centuries of developments, shaped significantly by numerous economists, political decisions, and pivotal historical moments. \\n\\nStill, it's conceivable that Marx’s views on centralized credit have influenced specific aspects of economic theories which have, in turn, indirectly sculpted portions of our modern banking structure. That being said, the majority of current central banks operate within a framework fundamentally grounded in capitalist principles, a system that Marx was inherently opposed to.\\n\\nMarx\"]],\n",
       " 'scores': [[1.0, 2.0], [2.0, 1.0], [2.0, 1.0], [2.0, 2.0]]}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distiset_complete[\"default\"][\"train\"][:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resources\n",
    "\n",
    "Additinally, you can find more information on the following resources.\n",
    "\n",
    "- [Distilabel 1.0 launch](https://argilla.io/blog/introducing-distilabel-1/)\n",
    "- [Datasets on the Hugging Face hub](https://huggingface.co/datasets?other=distilabel&sort=trending)\n",
    "- [Paper Implementations](https://distilabel.argilla.io/latest/sections/pipeline_samples/papers/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
